Literate programming
University of Luxembourg
Thursday, the 14th of November, 2024
Quarto: an open-source scientific and technical publishing system
Literate programming is the practice of mixing code and descriptive writing in order to execute and explain a data analysis simultaneously in the same document.
— William Landau Developer of {targets}
For a bioinformatician, pure programming is barely happening, we need reports/websites where content and formatting are disconnected
Source: {targets} user manual
Predecessor of Quarto.
Version 1 in 2016, the idea was to have text and code in one document, execute code and let the final markdown converted to different formats by pandoc.
Jupyter notebook were also exploring the same concept. Rendering order led sometimes to some frustration
Credit: Artwork by Allison Horst
<!DOCTYPE html>
<html>
<body>
<h1>This is a heading</h1>
<p>This is some text in a <b>paragraph</b></p>
<h2>This is a second level heading</h2>
<ul>
<li><a href="http://exa.com"><code>site</code></a>
<li><img src="https://images.computerhistory.org/revonline/images/500004391-03-01.jpg?w=200">
</ul>
</body>
</html>Extended Rmarkdown. Languages and Integrated Development Environment agnostic
.qmd) document to various formats (PDF, HTML…)knitrYAML header
Free text in Markdown
Code chunks
Code to be interpreted
knitr for jupyter for docx)How to add equations
Enclose in $ for in-line equations
$a^2+b^2=c^2$ renders as \(a^2+b^2=c^2\).
Double ($$) for separate equations.
$$G_{\mu v}=8 \pi G (T_{\mu v} + \rho _\Lambda \ g_{\mu v}) $$
renders as:
\[G_{\mu v}=8 \pi G (T_{\mu v} + \rho _\Lambda \ g_{\mu v})\]
shiny apps (using CI scheduled pipelines)Command to add an extensions among many available (+165 references)
dlsm-bioinfo/
├── bioinfo.qmd
├── dag-atac-seq.png
├── dag-rna-seq.png
├── insert-citation.png
├── orbi-bib-export.png
├── _extensions
│ └── quarto-journals
│ └── plos/
├── gaigneaux.bib
├── ginolhac.bib
├── references.bib
├── plos2015.bst <- bib style
├── _manuscript/ <- outputs
└── _quarto.yml <- main configDemo HTML output
PLoS PDF example:
Compiled here
git , as you do anyway right?Some icons from flaticon.com, credit to Valueinvestor
.github/workflows/publish.yml
on:
workflow_dispatch:
push:
branches: main
name: Quarto Publish
permissions:
contents: write
pages: write
jobs:
build-deploy:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Set up Quarto
uses: quarto-dev/quarto-actions/setup@v2
with:
version: pre-release
- uses: r-lib/actions/setup-r@v2
- uses: r-lib/actions/setup-r-dependencies@v2
- name: Publish to GitHub Pages (and render)
uses: quarto-dev/quarto-actions/publish@v2
with:
path: quarto-intro.qmd
target: gh-pages
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}.gitlab-ci.yml
image: alpine:latest
stages:
- build_image
- build_site
- deploy
build docker image:
stage: build_image
image: docker:20.10.12-dind
before_script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
script:
- 'docker pull $CI_REGISTRY_IMAGE:latest || true'
- docker build --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:latest .
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
- docker push $CI_REGISTRY_IMAGE:latest
only:
changes:
- Dockerfile
build_site:
stage: build_site
image: $CI_REGISTRY_IMAGE:latest
cache:
key: $CI_JOB_NAME
paths:
- _freeze/
- _site/
artifacts:
name: "$CI_JOB_NAME"
expire_in: 2 days
paths:
- _site/
before_script:
# https://mastodon.social/@eddelbuettel/111693310455494722
- |
Rscript -e "read.dcf('DESCRIPTION', 'Imports') |>
tools:::.split_dependencies() |>
names() |>
setdiff(tools:::.get_standard_package_names()$base) |>
install.packages()"
script:
- quarto render
when: always
Deploy to BASV53:
stage: deploy
only:
- main
dependencies:
- build_site
before_script:
- 'which ssh-agent || ( apk add --update openssh )'
- eval $(ssh-agent -s)
- 'which rsync || ( apk add --update rsync )'
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh && chmod 700 ~/.ssh
- 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- rsync -azh -e "ssh -p 8022" --delete --filter 'protect /preview' $CI_PROJECT_DIR/_site/ $SSH_HOST:/var/www/basv53/Dockerfile, thus 3 steps.Dockerfile
FROM rocker/r2u:22.04
RUN apt-get update && \
apt-get install -y \
curl netbase \
zip git \
libxml2-dev libcurl4-openssl-dev libmagick++-dev \
&& rm -rf /var/lib/apt/lists/*
RUN install.r tidyverse rmarkdown biobase knitr V8 gt gtextras ggimage ggrepel ggupset BiocManager && \
installBioc.r deseq2 apeglm
# from https://quarto.org/docs/get-started/
ARG QUARTO_VERSION="1.6.33"
RUN curl -LO https://github.com/quarto-dev/quarto-cli/releases/download/v${QUARTO_VERSION}/quarto-${QUARTO_VERSION}-linux-amd64.deb && \
apt-get update -qq && apt-get -y install \
./quarto-${QUARTO_VERSION}-linux-amd64.deb && rm quarto-${QUARTO_VERSION}-linux-amd64.deb \
&& apt autoremove -y && apt clean -y && rm -rf /var/lib/apt/lists/*r2u is awesome, offering packages as APT.
{tidyverse} install is < 10 secSummary
Quarto (qmd)Markdownrender to different output formatsFurther reading 📚
Acknowledgments 🙏 👏
Thank you for your attention!